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Fluid limit techniques have become a central tool to analyze 
queueing networks over the last decade, with applications to per- 
formance analysis, simulation and optimization. 

In this paper, some of these techniques are extended to a general 
class of skip-free Markov chains. As in the case of queueing models, a 
fluid approximation is obtained by scaling time, space and the initial 
condition by a large constant. The resulting fluid limit is the solution 
of an ordinary differential equation (ODE) in "most" of the state 
space. Stability and finer ergodic properties for the stochastic model 
then follow from stability of the set of fluid limits. Moreover, similarly 
to the queueing context where fluid models are routinely used to 
design control policies, the structure of the limiting ODE in this 
general setting provides an understanding of the dynamics of the 
Markov chain. These results are illustrated through application to 
Markov chain Monte Carlo methods. 



The use of ordinary differential equations (ODE) to analyze Markov chains 
was first suggested by Kurtz (1970). This idea was later refined by Newell 
(1982), who introduced the so-called fluid approximations with applica- 
tions to queueing networks. Since the 1990s, fluid models have been used 
to address delay in complex networks [Cruz (1991)] and bottleneck analysis 
[Chen and Mandelbaum (1991)]. The latter work followed an already exten- 
sive research program on diffusion approximations for networks [see Harrison 
(2000), Whitt (2002), Chen and Yao (2001) and the references therein]. 
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The purpose of this paper is to extend fluid limit techniques to a general 
class of discrete-time Markov chains {&k} ° n a ^-dimensional Euclidean 
state space X. Recall that a Markov chain is called skip-free if the increments 
(&k+i — are uniformly bounded in norm by a deterministic constant for 
each k and each initial condition. For example, Markov chain models of 
queueing systems are typically skip-free. Here, we consider a relaxation of 
this assumption in which the increments are assumed to be bounded in an 
L p -sense. Consequently, we find that the chain can be represented by the 
additive noise model 

(1) $ fe+ i = $ fc + A($ fe ) + e fc+ i, 

where {e^} is a martingale increment sequence w.r.t. the natural filtration 
of the process and A:X— >X is bounded. Associated with this chain, 

we consider the sequence of continuous-time processes 

77°(t;x) d =r~ 1 $u r i +aJ , rj?(t;0) =r~ 1 ^ = x, 

(2) 

r>0,a>0,xeX, 

obtained by interpolating and scaling the Markov chain in space and time. A 
fluid limit is obtained as a subsequential weak limit of a sequence {77" (■; x n )}, 
where {r n } and {x n } are two sequences such that lim n ^ 00 r n = 00 and 
lim ri _» 00 x n = x. The set of all such limits is called the fluid limit model. 
In queueing network applications, a fluid limit is easy to interpret in terms 
of mean flows; in most situations, it is a solution of a deterministic set of 
equations depending on network characteristics as well as the control policy 
[see, e.g., Chen and Mandelbaum (1991), Dai (1995), Dai and Meyn (1995), 
Chen and Yao (2001), Meyn (2007)]. The existence of limits and the conti- 
nuity of the fluid limit model may be established under general conditions 
on the increments (see Theorem 1.2). 

The fact that stability of the fluid limit model implies stability of the 
stochastic network was established in a limited setting in Malysev and Menc'sikov 
(1979). This was extended to a very broad class of multiclass networks by 
Dai (1995). A key step in the proof of these results is a multi-step state- 
dependent version of Foster's criterion introduced in Malysev and Menc'sikov 
(1979) for countable state space models, later extended to general state space 
models in Meyn and Tweedie (1993, 1994). The main result of Dai (1995) 
only established positive recurrence. Moments and rates of convergence to 
stationarity of the Markovian network model were obtained in Dai and Meyn 
(1995), based on an extension of Meyn and Tweedie (1994) using the sub- 
geometric /-ergodic theorem in Tuominen and Tweedie (1994) [recently ex- 
tended and simplified in work of Douc et al. (2004)]. Converse theorems have 
appeared in Dai and Weiss (1996), Dai (1996), Meyn (1995) that show that, 
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under rather strong conditions, instability of the fluid model implies tran- 
sience of the stochastic network. The counterexamples in Gamarnik and Hasenbein 
(2005), Dai et al. (2004) show that some additional conditions are necessary 
to obtain a converse. 

Under general conditions, including the generalized skip-free assumption, 
a fluid limit n is a weak solution (in a sense given below) to the homogeneous 
ODE 

(3) fi = h(fi). 

The vector field h is defined as a radial limit of the function A appearing in 
(1) under appropriate renormalization. 

Provided that the increments {e^} in the decomposition (1) are tight in 
L p , stability of the fluid limit model implies finite moments in steady state, 
as well as polynomial rates of convergence to stationarity; see Theorem 1.4. 

One advantage of the ODE approach over the usual Foster-Lyapunov 
approach to stability is that the ODE model provides insight into Markov 
chain dynamics. In the queueing context, the ODE model has many other 
applications, such as simulation variance reduction [Henderson et al. (2003)] 
and optimization [Chen and Meyn (1999)]. 

The remainder of the paper is organized as follows. Section 1.1 contains 
notation and assumptions, along with a construction of the fluid limit model. 
The main result is contained in Section 1.2, where it is shown that stability of 
the fluid limit model implies the existence of polynomial moments as well as 
polynomial rates of convergence to stationarity [known as (f,r)-ergodicity]. 

Fluid limits are characterized in Section 1.3. Proposition 1.5 provides 
conditions that guarantee that a fluid limit coincides with the weak solutions 
of the ODE (3). 

These results are applied to establish (/, r)-ergodicity of the random walk 
Metropolis-Hastings algorithm for super exponential densities in Section 2.1 
and sub exponential densities in Section 2.2. In Examples 2 and 4, the fluid 
limit model is stable and any fluid limit is a weak solution of the ODE (3), 
yet some fluid limits are nondeterministic. 

The conclusions contain proposed extensions, including diffusion limits of 
the form obtained in Harrison (2000), Whitt (2002), Chen and Yao (2001) 
and application of ODE methods for variance reduction in simulation and 
MCMC. 

1. Assumptions and statement of the results. 

1.1. Fluid limit: definitions. We consider a Markov chain 3> = f {&k}k>o 
on a (i-dimensional Euclidean space X equipped with its Borel sigma-field 
X . We denote by {J~k}k>o the natural filtration. The distribution of <& is 
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specified by its initial state &o = x £X and its transition kernel P. We write 
¥ x for the distribution of the chain conditional on the initial state <l>o = x 
and E x for the corresponding expectation. 

Denote by C(M + , X) the space of continuous X- valued functions on the infi- 
nite time interval [0, oo). We equip C(M + , X) with the local uniform topology. 
Denote by D(]R + ,X) the space of X- valued right-continuous functions with 
left limits on the infinite time interval [0, oo), hereafter cadlag functions. This 
space is endowed with the Skorokhod topology. For < T < +oo, denote by 
C([0, T],X) (resp. D([0, T],X)) the space of X-valued continuous functions 
(resp. cadlag functions) defined on [0, T], equipped with the uniform (resp. 
Skorokhod) topology. 

For x G X, a > and r > 0, consider the interpolated process 

(4) n?(t;x) d = r _1 $ Ltr i+aj, r/°(t;0) =r~ 1 <5> = x, 

where |_ - J stands for the lower integer part. Denote by Q^. x the image prob- 
ability on D(M + ,X) of Fx by rj"(-;x). In words, the renormalized process is 
obtained by scaling the Markov chain in space, time and initial condition. 
This is made precise in the following definition. 

Definition 1.1 (a- fluid limit). Let a > and x G X. A probability 
measure on D(M + ,X) is said to be an a- fluid limit if there exist se- 
quences of scaling factors {r n } C R+ and initial states {x n } C X satisfying 
lim n ^ oo v n — +oo and Xvs^n-^o^Xn — x such that ' - x } converges weakly 
to Q£ on D(R+,X) (denoted Q« ;a!n =>QS). 

The set {Q",x G X} of all such limits is referred to as the a- fluid limit 
model. An a-fluid limit is said to be deterministic if there exists a func- 
tion g £ D(]R + ,X) such that = 5 g , the Dirac mass at g. 

Assume that E^fl^ij] < oo for all i£X, where | • | denotes the Euclidean 
norm, and consider the decomposition 

(5) $ fc = * fc _i + A($ fc _i) + e fc , k>l, 
where 

(6) A(x) d =E x [<S> 1 -<S> ]=E x [<S> 1 }-x for all x G X, 

(7) e k d = - E[$ fc |JP fe _!] for all k > 1. 
In the sequel, we assume the following. 

Bl. There exists p>l such that lim^-_ >00 sup a , eX E x [|ei| p l{|ei| > K}] =0. 

B2. There exists (3 G [0,lA(p-l)) such that N(f3, A) d = sup a , eX {(l + \xf) x 
|A(x)|} < oo. 
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Theorem 1.2. Assume Bl and B2. Then, for allO <a< (3 and any se- 
quences {r n } C M+ and {x n } C X such that lim n _»oo r n = +oo and lim re _»oo x n = 
x, there exists a probability measure Q° on C(JR_)_,X) and subsequences {r nj } Q 
{r n } and {x nj } C {x n } such that _ =>■ Q". Furthermore, for all < 

a < /3, i/ie a-fluid limits are trivial in t/ie sense that = <5 g mi/i g(i) = x. 

Note that for any x £ X and < a < /3, we have Q"(j7,ry(0) = x) = 1, 
showing that x is the initial point of the fluid limit. 

1.2. Stability of fluid limits and Markov chain stability. There are several 
notions of stability that have appeared in the literature [see Meyn (2001), 
Theorem 3] and the surrounding discussion. We adopt the notion of stability 
introduced in Stolyar (1995). 

Definition 1.3 [Stability). The a-fluid limit model is said to be stable 
if there exist T > and p < 1 such that for any x £ X with \x\ = 1, 

(8) (Qg^e D(R +) X) !o m| T |r ? (i)| <p) = 1. 

Let / : X — > [1, oo) and L£) denote the vector space of all measurable func- 
tions g on X such that sup^gx IffC 37 )!//^) ^ s finite. L4> equipped with the 
norm \g\f = f sup xe x \g(x)\/ f(x) is a Banach space. 

Denote by || • \\f the /-total variation norm, defined for any finite signed 
measure v as |H|/ = sup| 9 |<j |f (g)|. 

We recall some basic definitions related to Markov chains on general state 
space; see Meyn and Tweedie (1993) for an in-depth presentation. A chain 
is said to be phi-irreducible if there exists a a-finite measure cj> such that 
J2 n >o P n (x, A) > for all x £ X whenever <j)(A) > 0. A set C £ X is v m -small 
if there exist a nontrivial measure v m and a positive integer m such that 
such that P m (x,-) > tc(x)u m (-). Petite sets are a generalization of small 
sets: a set C is said to be petite if there exists a distribution a on the posi- 
tive integers and a distribution v such that J2n>o a(n)P n (x, ■) > tc{x)u{-). 
Finally, an aperiodic chain is a chain such that the greatest common divisor 
of the set 

{m, Cis z/ m -small and v m = 8 m v for some S m > 0}, 

is one, for some small set C. For a phi-irreducible aperiodic chain, the petite 
sets are small [Meyn and Tweedie (1993), Proposition 5.5.7]. 

Let {r(n)} n£ N be a sequence of positive real numbers. An aperiodic phi- 
irreducible positive Harris chain with stationary distribution tt is called 
(f,r)-ergodic if 

lim r(n)\\P n (x,-) -ir\\ f =0 

tt — ^rvn * 
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for all x 6 X. If P is positive Harris recurrent with invariant probability ir, 

then the fundamental kernel Z is defined as Z = f (Id — P + II) -1 , where the 
kernel II is H(x, ■) = ir(-) for all x 6 X and Id is the identity kernel. For any 
measurable function g on X, the function g = Zg is a solution to the Pois- 
son equation, whenever the inverse is well defined [see Meyn and Tweedie 
(1993)]. 

The following theorem may be seen as an extension of [Dai and Meyn 
(1995), Theorem 5.5], which relates the stability of the fluid limit to the 
(/, r)-ergodicity of the original chain. 

Theorem 1.4. Let {<&k\k&n be a phi- irreducible and aperiodic Markov 
chain such that compact sets are petite. Assume Bl and B2 and that the 
(3-fluid limit model is stable. Then, for any 1 < q < (1 + (3)~ 1 p, 

(i) the Markov chain {&k}keN is (f^ q \r^)-ergodic with f^ q \x) = f 1 + 
\ x \ P -q{l+P) and r^(n) = n^ 1 ; 

(ii) the fundamental kernel Z is a bounded linear transformation from 
L* (q) to Lf {q ~ 1] 

1.3. Characterization of the fluid limits. Theorem 1.4 relates the ergod- 
icity of the Markov chain to the stability of the fluid limit and raises the 
question: how can we determine if the (3-fluid model is stable? To answer 
this question, we first characterize the set of fluid limits. 

In addition to assumptions B1-B2, we require conditions on the limiting 
behavior of the function A. 

B3. There exist an open cone C X \ {0} and a continuous function Aqq : 
— > X such that, for any compact subset H C 0, 

lim sup |r^|x|^A(rx) — A QO (x)| = 0, 
^+oo xgH 

where (3 is given by B2. 

The easy situation is when O = X\{0}, in which case the radial limit 
linv^oo rP\x\P A(rx) exists for Though this condition is met in exam- 

ples of interest, there are several situations for which the radial limits do 
not exist for directions belonging to some low-dimensional manifolds of the 
unit sphere. Let h be given by 

(9) MaO^M-^AooOr). 

A function /j,:I^X (where / C R + is an interval which can be open or 
closed, bounded or unbounded) is said to be a solution of the ODE (3) on / 
with initial condition x if [i is continuously differentiable on I for all t £ I, 
fj.(t) S 0, /i(0) = x and fi(t) = ho fj,(t). The following theorem shows that the 
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fluid limits restricted to evolve deterministically and, more precisely, that 
their supports on belong to the flow of the ODE. 



Proposition 1.5. Assume Bl, B2 and B3. For any < s < t, define 

(10) A(s,t) d = {r/G C(M + ,X):?7(u) G for allue[s,t]}. 

Then, for any x G X and any (3-fluid limit Q^, on A(s,i), 



sup 

s<u<t 



ri(u)-r](s) 



h o rj(v ) dv 



0. 



-a.s. 



Under very weak additional conditions, one may assume that the solu- 
tions of the ODE (3) with initial condition x G exist and are unique on 
a nonvanishing interval [0,T X ]. In such a case, Proposition 1.5 provides a 
handy description of the fluid limit. 

B4. Assume that for all x G 0, there exists T x > such that the ODE (3) 
with initial condition x has a unique solution, denoted n(-;x) on an 
interval [O,?^]. 

Assumption B4 is satisfied if Aoo is locally Lipschitz on 0; in such a 
case, h is locally Lipschitz on and it then follows from classical results 
on the existence of solutions of the ODE [see, e.g., Verhulst (1996)] that for 
any x G 0, there exists T x > such that, on the interval [0,T X ], the ODE 
(3) has a unique solution /i with initial condition /i(0) = x. In addition, if 
the ODE (3) has two solutions, \x\ and /i2, on an interval I which satisfy 
/xi (io) = ^(to) = xo for some to G /, then ni(t) = fJ-2(t) for any t £ I. 

An elementary application of Proposition 1.5 shows that under this addi- 
tional assumption, a fluid limit starting at xq G coincides with the solution 
of the ODE (3) with initial condition xo on a nonvanishing interval. 

Theorem 1.6. Assume B1-B4. Let x G 0. There then exists T x > 
such that = 5^.. x) on D([0,T x ],X). 

As a corollary of Theorem 1.6, we have the following. 

Corollary 1.7. Assume that = X \ {0} in B3. Then all (3-fluid limits 
are deterministic and solve the ODE (3). Furthermore, for any e > and 
x G X, and any sequences {r n } C M+ and {x n } C X such that lim ra _ too r n = 
+oo and limn^oo x n = x, 

limP^f sup \r]P(t;x n )-fi(t;x)\>e)=0. 

n \0<t<T x J 
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Hence, the fluid limit depends only on the initial value x and does not 
depend upon the choice of the sequences {r n } and {x n }. 

The last step is to relate the stability of the fluid limit [see (8)] to the 
behavior of the solutions of the ODE, when such solutions are well defined. 
From the discussion above, we may deduce a first elementary stability con- 
dition. Assume that B3 holds with = X \ {0}. In this case, the fluid limit 
model is stable if there exist p < 1 and T < oo such that, for any \x\ = 1, 
inf[ 0j r] x)| < p, that is, the solutions of the ODE enter a sphere of radius 
p < 1 before a given time T. 

Theorem 1.8. Let {^fcjfceN be a phi-irreducible and aperiodic Markov 
chain such that compact sets are petite. Let p, < p < 1 and T > 0. Assume 
that B1-B4 hold with = X \ {0}. Assume, in addition, that for any x 
satisfying \x\ = 1, the solution p(-;x) is such that inf[ 0i TA7y |M"> x )\ — P- 
Then, the f3 -fluid limit model is stable and the conclusions of Theorem 1.4 
hold. 

When B3 holds for a strict subset of the state space C X \ {0}, the 
situation is more difficult because some fluid limits are not solutions of the 
ODE. Regardless, under general assumptions, stability of the ODE implies 
stability of the fluid limit model. 

Theorem 1.9. Let {&k}keN be a phi- irreducible and aperiodic Markov 
chain such that compact sets are petite. Assume that B1-B4 hold with C 
X\{0}. Assume, in addition, that: 

(i) there exists Tq > such that for any x, \x\ = 1, and for any (5 -fluid 
limit Qg, 



(ii) for any K > 0, there exist Tr > and < px < 1 such that for any 
x £ 0, \x\ < K, 



(11) 




(12) 




(iii) 



for any compact set H C O and any K , 



Sl H a = {H(t;x):x £H,te[0,T x AT K ]} 



is a compact subset ofO. 



Then, the (3-fluid model is stable and the conclusions of Theorem 1.4 hold. 
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Condition (i) implies that each /3-fluid limit reaches the set in a finite 
time. When the initial condition does belongs to 0, this condition is 

automatically fulfilled. When x does not belong to 0, this condition typically 
requires that there is a force driving the chain into 0. The verification of 
this property generally requires some problem-dependent and sometimes 
intricate constructions (see, e.g., Example 2). Condition (ii) implies that the 
solution n(-;x) of the ODE with initial point x S reaches a ball inside 
the unit sphere before approaching the singularity. This also means that the 
singular set is repulsive for the solution of the ODE. 

2. The ODE method for the Metropolis Hastings algorithm. The Metro- 
polis-Hastings (MH) algorithm [see Robert and Casella (2004) and the ref- 
erences therein] is a popular computational method for generating samples 
from virtually any distribution ir. In particular, there is no need for the nor- 
malizing constant to be known and the space X = M. d (for some integer d) 
on which it is defined can be high-dimensional. The method consists of sim- 
ulating an ergodic Markov chain {&k}k>o on X with transition probability 
P such that 7r is the stationary distribution for this chain, that is, ttP = w. 

The MH algorithm requires the choice of a proposal kernel q. In order 
to simplify the discussion, we will here assume that it and q admit den- 
sities with respect to the Lebesgue measure A Leb , denoted (with an abuse 
of notation) ir and q hereafter. We denote by Q the probability defined 
by Q(A) = J^q(y)X Lch (dy). The role of the kernel q consists of proposing 
potential transitions for the Markov chain Given that the chain is 

currently at x, a candidate y is accepted with probability a(x,y), defined as 
a(x,y) = 1 A ~|fy • Otherwise it is rejected and the Markov chain stays 
at its current location x. The transition kernel P of this Markov chain takes 
the form, for x £ X and A € B(X), 



(13) 



P{x, A) = / a{x, x + y)q{x, x + y)X Leb (dy) 

JA-x 

+ t A {x) f {1 - a{x, x + y)}q(x, x + y)X Lch (dy), 

JX-x 



where A — x = f {y G X, x + y G A}. The Markov chain P is reversible with 
respect to ir and therefore admits tt as invariant distribution. For the purpose 
of illustration, we focus on the symmetric increments random walk MH 
algorithm (hereafter SRWM), in which q(x,y) = q{y — x) for some symmetric 
distribution q on X. Under these assumptions, the acceptance probability 
simplifies to a(x, y) = 1 A \jr{y)/ix{x)\ . For any measurable function W : X — > 
X, 

E x \W($i)]-W(x)= j {W(x + y)-W(x)}q(y)X Lch (dy) 
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+ / {W(x + y)-W(x)} 



J R x 



7t(x) 



q(y)\ Lch (dy) 



where A x = {y G X, ir(x + y) > vr(x)} is the acceptance region (moves toward 

x + A x are accepted with probability one) and = f X \ is the potential 
rejection region. From Roberts and Tweedie (1996), Theorem 2.2, we obtain 
the following basic result. 

Theorem 2.1. Suppose that the target density ir is positive and continu- 
ous and that q is bounded away from zero, that is, there exist 5 q > and e q > 
such that q(x) > e q for \x\ < 5 q . Then, the random-walk-based Metropo- 
lis algorithm on {X,^} is A Lcb -irreducible, aperiodic and every nonempty 
bounded set is small. 

In the sequel, we assume that q has a moment of order p > 1. To apply 
the results presented in Section 1, we must first compute A(x) = E^f^i] — x, 
that is, to set W(x) = x in the previous formula. Since q is symmetric and 
therefore zero-mean, the previous reduces to 



Note that, for any x G X, |ei| < l^i — <l?o| + rn¥ x -&.s., where m = f \y\q(y) x 
X Lch (dy). Therefore, for any K > 0, 

E a .[|ei|fl{|ei| > K}] < 2 P E X [(|$ 1 - $ | p + m p )l{|$! - * | > K - m}] 



showing that assumption Bl is satisfied as soon as the increment distribution 
has a bounded pth moment. Because, on the set R^, ir(x + y) < n(x), we 
similarly have |A(se)| < / \y\q(y) X Leh (dy) showing, that B2 is satisfied with 
f3 = 0; nevertheless, in some examples, for (3 = 0, Aqo can be zero and the 
fluid limit model is unstable. In these cases, it is necessary to use larger (5 
(see Section 2.2). 

2.1. Superexponential target densities. In this section, we focus on target 
densities ir on X which are superexponential. Define nix) = x/\x\. 

Definition 2.2 (Superexponential p.d.f.). A probability density func- 
tion 7r is said to be superexponential if ir is positive, has continuous first 
derivatives and lim| a ,|^ 00 (n(a;),£(x)) = — oo, where i(x) = Vlog7r(x). 



(14) 
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The condition implies that for any H > 0, there exists R > such that 
(15) vr(x + an(x)) for |z| > i?, a > 0, 

that is, 7r(x) is at least exponentially decaying along any ray with the rate 
H tending to infinity as |x| goes to infinity. It also implies that for x large 

enough, the contour manifold C x = f {y £ X, tt(x + y) = vr(x)} can be param- 
eterized by the unit sphere S since each ray meets C x at exactly one point. 
In addition, for sufficiently large \x\, the acceptance region A x is the set 
enclosed by the contour manifold C x (see Figure 1). Denote by A B the 
symmetric difference of the sets A and B. 

Definition 2.3 {q-radial limit). We say that the family of rejection 
regions {R rx ,r > 0, x £ 0} has q-radial limits over the open cone C X\ {0} 
if there exists a collection of sets {Roo,x,% £ 0} such that, for any compact 
subset H C 0, linv^oo sup xgH Q(R rx © Roo,x) = 0. 

Proposition 2.4. Assume that the target density tt is super-exponential. 
Assume, in addition, that the family {R rx ,r > 0,x £ 0} has a q-radial limit 
over an open cone O C X \ {0}. Then, for any compact set H C O, 

lim^ooSup^gH |A(rx)- Aoo(x)| = 0, where A^x) d = -J Roo x yq(y)X Lch (dy). 

The proof is postponed to Section 5.1. The definition of the limiting field 
Aqo becomes simple when the rejection region radially converges to a half- 
space. 

Definition 2.5 (q-regularity in the tails). We say that the target den- 
sity 7r is q-regular in the tails over if the family {R rx ,r > 0,x £ 0} has 




Fig. 1. 
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g-radial limits over an open cone C X \ {0} and there exists a continuous 
function : X \ {0} — ► X such that, for all x € 0, 

(16) Q(Roo, x e {y e x, ( y , i^x)) < o}) = o. 

Regularity in the tails holds with £qo(x) = lim^oo n(£(rx)) when the curva- 
ture at of the contour manifold C rx goes to zero as r — ► oo; nevertheless, 
this condition may still hold in situations where there exists a sequence {x n } 
with lim \x n \ = oo such that the curvature of the contour manifolds C Xn at 
zero can grow to infinity (see Examples 1 and 2). Assume that 

(17) q(x)=det~ 1 / 2 (Z)q (Z~ 1 / 2 x), 

where E is a positive definite matrix and qo is a rotationally invariant dis- 
tribution, that is, qo(Ux) = qo(x) for any unitary matrix U, and is such that 



y 2 My)^ ch {dy) < oo. 

X 



Proposition 2.6. Assume that the target density tt is super-exponential 
and g-regular in the tails over the open cone C X\ {0}. Then, the SRWM 
algorithm with proposal q given in (17) satisfies assumption B3 on O with 

(18) A oo (x)=m 1 (q ) 



£*oo(aOI 

where is defined in (16) and mi (go) = f Jxyi^-{y 1 >o}Qo{y)^ Leh (dy) > 0, 
where y = (y u . . . , y d ). 



The proof is given in Section 5.1. If E = Id and ^oo(s) = lim r ._ KX ,n(^(r'j;)), 
then the ODE may be seen as a version of steepest ascent algorithm to 
maximize log7r. It may appear that convergence would be faster if mi(qo) 
is increased. While it is true for the ODE, we cannot reach such a positive 
conclusion for the algorithm itself because we do not control the fluctuation 
of the algorithm around its limit. 



2.1.1. Regular case. The tail regularity condition and the definition of 
the ODE limit are more transparent in a class of models which are very 
natural in many statistical contexts, namely, the exponential family. Follow- 
ing Roberts and Tweedie (1996), define the class V as consisting of those 
everywhere positive densities with continuous second derivatives tt satisfying 

(19) ir(x) oc g{x) exp{— p(x)}, 



where: 
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• g is a positive function slowly varying at infinity, that is, for any K > 0, 

/ on N ,. ■ f g(x + y) v g(x + y) , 

(20) limsup mt — = limsup sup — — = 1; 

|a!|-oo \V\< K 9{X) \ x \^oo \y\<K 9( x ) 

• p is a positive polynomial in X of 6vgii order ui and. linii^i >OQ Pm (x) = +oo, 

where p m denotes the polynomial consisting only of the p's mth order 
terms. 



Proposition 2.7. Assume that tt e V and let q be given by (17). Then, 
7r is super- exponential, q-regular in the tails over X\{0} with £oo( x ) = 
—n[X7p m (n(x))]. For any x E X\ {0}, there exists T x > such that the ODE 
fi = Aoo(^) with initial condition x and Aoo given by (18) has a unique so- 
lution on [0,T X ) and lim 4 ^ T - [i(t;x) = 0. In addition, the fluid limit Q x is 
deterministic on D([0, T x ], X), with support function fi(-;x). 

The proof is skipped for brevity [see Fort et al. (2006)]. Because all 
the solutions of the initial value problem (i = — mi(go)v / S"-[v / SVp m (n(//))], 
/i(0) = x are zero after a fixed amount of time T for any initial condition 
on the unit sphere, we may apply Theorem 1.8. We have, from Theorem 2.1 
and Theorem 1.8, the following. 

Theorem 2.8. Consider the SRWM Markov chain with target distribu- 
tion 7r G V and increment distribution q having a moment of order p > 1 
and satisfying (17). Then, for any 1 < u < p, the SRWM Markov chain is 
(fu,r u )-ergodic with 




-10-5 5 10 -30 -20 -10 10 20 30 



Fig. 2. Contour curves of the target densities (21) (left panel) and (26) with 5 — 0.4 
(right panel). 
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i 3 4 5 6 10 11 12 13 14 15 16 



Fig. 3. Grey lines: A; Black lines: Aoo for the target densities (21) (left panel) and (26) 
with 5 = 0.4 (right panel). 

Example 1. To illustrate our findings, consider the target density, bor- 
rowed from Jarner and Hansen (2000), Example 5.3, 

(21) vr(xi, X2) oc (1 + xf + x\ + x\x^) exp( — (xf + x\)). 

The contour curves are illustrated in Figure 2. They are almost circular 
except from some small wedges by the x-axis. Due to the wedges, the 
curvature of the contour manifold at (x,0) is (x 6 — l)/x and therefore 
tends to infinity along the x-axis [Jarner and Hansen (2000)]. Since n G 

1.2 12 




0,2 0.4 0.6 0.B 1 1,2 ~°-l,2 0.2 0.4 0.6 0,8 1 1,2 

Fig. 4. Dotted lines: trajectories of the interpolated process (2) for the random walk 
Metropolis-Hastings (SRWM) algorithm for a set of initial conditions on the unit sphere 
in (0,7r/2) for the target densities (21) (left panel) and (26) (right panel); Solid lines: flow 
of the associated ODE. 
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Fig. 5. Contour plot of the target densities (22) (left panel) and (27) (right panel). 

V, Proposition 2.7 shows that tt is super-exponential, regular in the tails 
and ^oo(^) = —n(x). Taking q ~ J\f(0, <r 2 Id), A oa (x) = — an(x) / \^2n and the 
(Caratheodory) solution of the initial value problem fi = A 00 (/i), fj,(0) = x is 
given by n{t;x) = (\x\ — at / y/2ir)l{<jt < \/27r|^|}^/|^| • Along the sequence 

{xk = (k, ±/c~ 4 )}fc>i, the normed gradient n[£(xk)] converges to (0, ±1), 
showing that whereas £oc is the radial limit of the normed gradient n[£] (i.e., 
for any u G S, liniA^oo n[£(Xu)] =£oo(u)), limsup^i^.^ |n[^(x)] - £oo(x)\ = 2. 
Therefore, the normed gradient n[£(x)] does not have a limit as |x| — > oo 
along the a>axis. Nevertheless, the fluid limit exists and is extremely sim- 
ple to determine. Hence, the ergodicity of the SRWM sampler with target 
distribution (21) may be established [note that for this example, the theory 
developed in Roberts and Tweedie (1996) and in Jarner and Hansen (2000) 
does not apply]. The functions A and Aoo are displayed in Figure 3. The 
flow of the initial value problem /i = A^ (fi) for a set of initial conditions on 
the unit sphere in (0,7r/2) is displayed in Figure 4. 

2.1.2. Irregular case. We give an example for which, in Proposition 2.4, 
OCX\{0}. 

Example 2. In this example [also borrowed from Jarner and Hansen 
(2000)], we consider the mixture of two Gaussian distributions on M 2 . For 
some a 2 > 1 and < a < 1 , set 

(22) vr(x) oc aexp(-(l/2)xT 1 ^ 1 x) + (1 - a)exp(-(l/2)x'T^ 1 x), 

where T^ 1 = f diag(a 2 ,l) and T^ 1 = diag(l, a 2 ). The contour curves for tt 
with a = 4 are illustrated in Figure 5. We see that the contour curves have 
some sharp bends along the diagonals that do not disappear in the limit, 
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2 3 4 5 8 10 12 14 15 18 20 



Fig. 6. Grey lines: A; Black lines: Aoo for the target density (22) (left panel) and (27) 
(right panel). 

even though the contour curves of the two components of the mixtures are 
smooth ellipses. Equation (51) of Jarner and Hansen (2000), indeed shows 
that the curvature of the contour curve on the diagonal tends to infinity. 
As shown in the following lemma, however, this target density is regular in 
the tails over = X \ {x = (xi,X2) € M 2 , \x\\ = \x2\} (and not over X \ {0}). 
More precisely, we have the following. 

1.2 1.2r 




~°$2 Ol M 06 oti 1 \2 ~°4>,2 02 0.4 0.6 0.8 1 1.2 

Fig. 7. Dotted lines: interpolated process for a set of initial conditions on the unit sphere 
for the target density (22) (left panel) and (27) (right panel); Solid lines: flow of the initial 
value problem fi = h(fi) with h(x) — \x\~ f3 A oc (x) ; (3 = and Aoo are given by Lemma 2.9 
(left panel) and /3, are given by Lemma 2.16 (right panel). 
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Lemma 2.9. For any e > 0, there exist M and K such that 
(23) sup |A(a:)- Aoo(x)| <e, 

\x\>K,\\xi\ — \x2\\>M 

where A 0O (x) d ^ f -/ t Rao ^ (y)yq(y) X Lch (dy) with R^ x d ^ {y, (y,T^x) > 0} 
*/ Nil > l 3 ^! and Roo,x = f {y, (y> r^ 1 ^) > 0} otherwise. 

The proof is postponed to Section 5.2. Since q satisfies (17), when £ = 
Id, for any x G 0, we have either Aoo(x) = — c g n(r^" 1 x) if |xi| > \x2\ or 
Aoo(^) = — c q n(T~[ x) if |xi| < | o?2 1 , where c q is a constant depending on the 
increment distribution q. This is illustrated in Figure 6, which displays the 
functions A and Aoo and shows that these two functions are asymptotically 
close outside a band along the main diagonal. The flows of the initial value 
problem fi = Aoo(/i) for a set of initial conditions in (0, vr/2) are displayed 
in Figure 7. 

We now prove that Theorem 1.9 applies. Conditions B1-B2 hold, as dis- 
cussed above. Condition B3 results from Lemma 2.9. It remains to prove 
that B4 and conditions (i)-(iii) are verified. The proof of condition (i) is 
certainly the most difficult to check in this example. 

Proposition 2.10. Consider the SRWM Markov chain with target dis- 
tribution given by (22). Assume that q is rotationally invariant and with 
compact support. Then, B4 as well as conditions (i), (ii) and (iii) of Theo- 
rem 1.9 hold. 



0.8 r 8 




-0.1 0,1 0,2 0.3 OA 0.5 0.6 0.7 -0,1 01 0,2 0.3 0,4 0,5 0,6 0,7 



Fig. 8. Dotted lines: trajectories of the interpolated process (2) for the SRWM with target 
density (22) (left panel) and (27) (right panel) and initial condition (l/\/2, 1/V2); Solid 
lines: flow of the associated ODE. 
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A detailed proof is provided in Section 5.2. Note that the fluid limit model 
is not deterministic in this example: for x on the diagonal in X, the support of 
the fluid limit Q° consists of two trajectories, each of which are solutions of 
the ODE. This is illustrated in Figure 8. By Theorem 1.9 and the discussion 
above, we may conclude that if the increment distribution q is compactly 
supported, then the SRWM Markov chain with target distribution ir given 
by (22) is (/„, r s )-ergodic with f u (x) = 1 + \x\ u and r s (t) ~ t s for any u > 
and s > 0. 

2.2. Subexponential density. In this section, we focus on target densities 
7r on X which are subexponential. We assume that q satisfies (17) and has 
moment of order p>2. This section is organized as above: we start with the 
regular case (Example 3) and then consider the irregular case (Example 4). 

Definition 2.11 (Subexponential p.d.f.). A probability density func- 
tion 7r is said to be subexponential if tt is positive with continuous first deriva- 
tives, (n(x) , n(£(x))) < for all sufficiently large x and lim^i^oo \£(x)\ = 0. 

The condition implies that for any R < oo, lim|. E |^ 00 sup\ y \ <R 7r(x + y)/ir(x) - 
1, which implies that lim^i^oo | A(x)| =0. Subexponential target densities 
provide examples that require the use of positive (5 in the normalization to 
obtain a nontrivial fluid limit model. 

The condition {n(x),n(£(x))) < for all sufficiently large |x| implies that 
for e small enough, the contour manifold C e can be parameterized by the 
unit sphere (see the discussion above) and that for sufficiently large |x|, the 
acceptance region A x is the set enclosed by the contour manifold C x (see 
Figure 1). 

Definition 2.12 [g-regularity in the tails (subexponential)]. We say 
that 7r is q-regular in the tails over an open cone C X \ {0} if there exists 
a continuous function : — > X and (3 G (0,1) such that, for any compact 
set H C O and any K > 0, 

hm sup / rP\xf{ *( r *\ V) - l) - <M*),2/> q(y)\ Lch (dy) = 0, 

lim supQ(R rx Q{y,(e oo (x),y)>0})=0. 

Proposition 2.13. Assume that the target density ir is subexponential 
and q-regular in the tails over an open cone O C X \ {0} and that q satis- 
fies (17). Then, for any compact set H C O, lim^oo sup xGH |r^|x|^A(r3;) — 
Aoo(^)| = with 

AooCx) = / y{£ oo (x),y)q(y)\ Lch (dy)=m 2 (q )Zt oo (x), 
J{y,{£oo(x),y)>0} 
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where m 2 {q ) = f x yjt{ yi > }q {y)X Lch {dy) > 0. 

The proof is similar to Proposition 2.4 and is omitted for brevity. Once 
again, if the curvature of the contour curve goes to zero at infinity, then 
^oo(x) is, for large x, asymptotically colinear to n[Vlog7r(x)]. However, 
whereas |Vlog7r(x)| — > as \x\ — > oo, the renormalization prevents ioo{x) 
from vanishing at oo; on the contrary, it converges radially to a constant 
along each ray. As above, the tail regularity condition may still hold, even 
when the curvature goes to infinity; see Example 3. As above, the subexpo- 
nential tail regularity condition and the definition of the ODE limit are more 
transparent in the Weibullian family. Mimicking the construction above, de- 
fine, for 5 > 0, the class Vs as consisting of those everywhere positive densi- 
ties with continuous second derivatives tt satisfying 

(24) tt(s) <xg(x)exp{-p s (x)}, 

where g is a positive function slowly varying at infinity [see (20)] and p is a 
positive polynomial in X of even order m with lim| x |^ 00 p m (x) = +oo. 

PROPOSITION 2.14. Assume that tt eVs for some < 5 < 1/m and let 
q be given by (17). Then, it is subexponential and q-regular in the tails with 
[3=1 — m5 and loo{x) = —5p^ 1 (n(x))Vp m (n(x)). For any x € X\ {0}, there 
exists T x > such that the ODE jj, = h(fi) with initial condition x and h given 
by 

(25) h{x) = -8\x\- { ~ 1 ~ m ^m 2 {qo)p & -\n{x))T l Vp m {n{x)) 

has a unique solution on [0,T X ) and lim^ T - fj,(t; x) =0. In addition, the 
fluid limit Q^. is deterministic on D([0,T x ],X), with support function /j,(-;x). 

We may apply Theorem 1.8. From Theorem 2.1 and Proposition 2.14 we 
have the following. 

Theorem 2.15. Consider the SRWM Markov chain with target distri- 
bution tt on Vs and increment distribution q having a moment of order p>2 
and satisfying (17). Then, for any 1 < u < p/(2 — mS), the SRWM Markov 
chain is (f u ,r u )-ergodic with 

f u (x) = 1 + \xr u ^~ m5 \ r u (t) ~ t»-\ 

Example 3. Consider the subexponential Weibullian family derived 
from Example 1, 

(26) 7r(xi,x 2 ) oc (1 + x\ + xl + x 8 1 xl) s exp(-(xj + xl) 5 ). 
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The contour curves are displayed in Figure 2. Since ir G Vs, Proposition 
2.14 shows that ir is sub exponential and regular in the tails with (3 = 1 — 25 
and (.oo{x) = —25n(x). Taking q ~ M(0, cr 2 Id), Aoc(x) = -a 2 5n(x) and the 
(Caratheodory) solutions of the initial value problem fi = l/il"^ 1 " 25 ) A oc (/x), 
/i(0) = x are given by n(t;x) = [\x\ 2 ^~^ - 2a 2 S(l - fy] ^ 1 ' 5 )' 1 n(x) x 
lu|2Ci-«)_2o- 2 5(i-5)t>o- Here, again, the gradient (even properly normal- 
ized) does not have a limit as |x| — ► oo along the x-axis, but the fluid limit 
model is simple to determine. Hence, the ergodicity of the SRWM sampler 
with target distribution (26) may be established [note that for this example, 
the theory developed in Fort and Moulines (2003) and Douc et al. (2004) 
does not apply]. The functions A and A^ are displayed in Figure 3. The 
flow of the initial value problem fi = h{jj) for a set of initial conditions on 
the unit sphere in (0,tt/2) is displayed in Figure 4, together with trajectories 
of the interpolated process. 

Example 4. Consider the mixture of bivariate Weibull distributions [see 
Patra and Dey (1999) for applications], 

tt{x) oc a(x'T7 1 x) <5 " 1 exp(-(l/2)(x'r^ 1 x) 5 ) 

(27) 

+ (1 - a){x'T^x) 5 - 1 exp(-(l/2)(xT 2 " 1 x) 5 ), 

where Tj, i = ±1,2, are defined in Example 2 and < a < 1. Similarly to 
Example 2, the curvature of the contour curve on the diagonal tends to 
infinity; nevertheless, the target density is regular in the tails over = 
X \ {x = (x\,X2) £ M 2 , = \x2\}. More precisely, we have the following. 

Lemma 2.16. For any e > 0, there exist M and K such that 

(28) sup | |s|^A(x) - Aoo (a;) | < e, 

\x\>K,\\xi\-\x2\\>M 

where (3 = f 1 — 25 and Aoo(x) = f — m2(qo)\x\P 5(x'F2 1 x) s ~ 1 T l T2 1 x if \x±\ > 
\x$\ and A oa (x)= t —m2(qo)\x\^5(x'T^ 1 x) s ^ 1 T,T^ 1 x otherwise. 

We can then establish the analog of Proposition 2.10 for the target dis- 
tribution (27), again assuming that the proposal distribution q has compact 
support. The details are omitted for brevity. From the discussions above, 
the SRWM Markov chain with target distribution ir given by (27) is (f u , r s )- 
ergodic with f u (x) = 1 + \x\ u and r s (t) ~ t s for all u > 0, s > 0. 

3. Conclusions. ODE techniques provide a general and powerful ap- 
proach to establishing stability and ergodic theorems for a Markov chain. In 
typical applications, the assumptions of this paper hold for any p > and, 
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consequently, the ergodic Theorem 1.4 asserts that the mean of any func- 
tion with polynomial growth converges to its steady-state mean faster than 
any polynomial rate. The counterexample presented in Gamarnik and Meyn 
(2005) shows that, in general, it is impossible to obtain a geometric rate of 
convergence, even when A, {e^} and the function / are bounded. 

The ODE method developed within the queueing networks research com- 
munity has undergone many refinements and has been applied in many very 
different contexts. Some of these extensions might serve well in other appli- 
cations, such as MCMC. In particular, we should note the following points. 

(i) Control variates have been proposed previously in MCMC to speed 
convergence and construct stopping rules [Robert (1998)]. The fluid model 
is a convenient tool for constructing control variates for application in the 
simulation of networks. The resulting simulators show dramatic performance 
improvements in numerical experiments: a hundredfold variance reduction 
is obtained in experiments presented in Henderson and Meyn (1997) and 
Henderson et al. (2003) based on marginal additional computational effort. 
Moreover, analytical results demonstrate that the asymptotic behavior of 
the controlled estimators are greatly improved [Meyn (2005, 2006, 2007)]. 
It is likely that both the theory and methodology can be extended to other 
applications. 

(ii) A current focus of interest in the networks community is the re- 
flected diffusion model obtained under a "heavy traffic scaling." An analog 
of "heavy-traffic" in MCMC is the case (3 > considered in this paper; the 
larger scaling is necessary to obtain a nonstatic fluid limit (see Theorem 
1.2). We have maintained /3 < 1 in order to obtain a deterministic limit. 
With (3 = 1, we expect that a diffusion limit will be obtained for the scaled 
MH algorithm under general conditions. This will be an important tool in 
the subexponential case. In the fluid setting of this paper, when (3 > 0, it 
is necessary to assume a great deal of regularity on the densities tt and q 
appearing in the MH algorithm to obtain a meaningful fluid limit model. We 
expect that very different regularity assumptions will be required to obtain 
a diffusion limit and that new insights will be obtained from properties of 
the resulting diffusion model. 

4. Proofs of the main results. 

4.1. State- dependent drift conditions. In this section, we improve the 
state-dependent drift conditions proposed by Filonov (1989) for discrete 
state space and later extended by Meyn and Tweedie (1994) for general 
state space Markov chains [see also Meyn and Tweedie (1993) and Robert 
(2000) for additional references and comments]. 

Following Nummelin and Tuominen (1983), we denote by A the set of 
nondecreasing sequences r = {r(n)} nG N satisfying lirm^oo [ log r(n)/n = 0, 



22 



G. FORT, S. MEYN, E. MOULINES AND P. PRIOURET 



that is, log r(n)/n converges to zero monotonically from above. A sequence 
r G A is said to be subgeometric. Examples include polynomial sequences 
r(n) = (n + l) s with 5 > and truly sub exponential sequences, r(n) = (n + 
l) s e cnl [c> and 7 G (0, 1)]. Denote by C the set of functions 



C 



def 



(29) 



|^:[l,oo) 



differentiable and inf 

{«e[i,oo)} 

def 



is concave, monotone nondecreasing, 
v)>0, lim <f>'(v) =0 



For G C, define H^(v) = J ± (l/(p(x)) dx. The function if^ : [1, 00) — ► [0, 00) 
is increasing and Mm. v ^ 00 H ( j ) {v) = 00; see [Douc et al. (2004), Section 2]. 

Define, for u > 0, r^iu) = f o HT^{u)/ip o iLT^O), where -f^" 1 is the in- 
verse of i?</>- The function u 1— > r^{u) is log-concave and thus the sequence 
{r (/ ,(/c)}/ c >o is subgeometric. Polynomial functions V 9 ( t; ) = v a , a G (0,1) are 
associated with polynomial sequences r v (k) = (1 + (1 — a)^) ^ 1 " ) . 

Proposition 4.1. Lei /:X— >• [l,oo) and F:X — > [l,oo) 6e measurable 
functions, e G (0, 1) &e a constant and C £ X be a set. Assume that sup c f /V < 
00 and that there exists a stopping time r > 1 such that, for any x ^ C , 

-1 



(30) 



E, 



.k=0 



<V(x) and E x [V(<f> T )] < (1 - e)V{x). 



Then, for all x i C, E ffi [£^ /(**)] < (e" 1 V sup (7 //V r )F(x). //, 
in addition, we assume that sup 2 . GC {/(x) + E^V^i)]} < 00, then 



Proof. Setf = f 



r T n| by r d jf 0; T l d -5. f ^ def _^ n _ x 



t±cc(&o) + lc(^o) an( i define recursively the sequence 
f and r n = f r"" 1 + f o r ™ \ where 8 is the shift 
operator. For any n G N, define by <l> n = the chain sampled at the in- 
stants {r n } n >o- {^n}n>o is a Markov chain with transition kernel P(x, A) = 
Pj;($f G A), x G X, AG Af. Equation (30) implies that 

(31) PV{x)=E x [V{<f> f )}<V(x)- F(x) for all x $C, 

where F(x) d = sE^z] f($ k )]. Let f c = inf{n > l,$ n G C}. Applying the 
Markov property and the bound tq < r 7 " , we obtain, for all x G" C, 

frc — 1 f o6' fc 

£/(<!>* 

Lfc=o 



E, 



<E, 



E E /(* 



fc=0 jf=0 

"TC — 1 



E 

L fc=o 



+ 



+ E iC [/(cD T , c )l {r , c<oo} ] 



sup 

C* v 



E x [l/($ T , c )l {r , c<oo} ]. 
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Furthermore, (31) and the comparison theorem [Meyn and Tweedie (1993), 
Theorem 11.3.2] applied to the sampled chain {<3> n } n >o yields 



E, 



TQ -l 

E F (^) 

. k=0 



+ E x [V(^ c )t {T , c<oo} ]<V(x), xiC, 



which concludes the proof of the first claim. The second claim follows by 
writing, for x £ C, 



E, 



TQ 



£/(**) 

.fe=0 



<2sup/ + E a 

c 



TO 



k=l 



< 2sup/ + E ; 

c 



TO — l 

E /(**) 

L k=0 



1{X 1 £ C}E Xl 
<2sup/+ (V 1 Vsnpf/V^E x [t{X 1 £C}V(X 1 )}. n 



Proposition 4.2. Assume that the conditions of Proposition 4.1 
are satisfied with f(x) = (ft o V(x) for x ^ C with (ft £ C. Then, for x ^ C , 
^[ElZo 1 r 4,( k )\ < M~ x V(x) and sup^E^E^" 1 r^k)] < oo, where, 



(32) for all t 0(t) d = <f>(Mt) and M d = 



e 1 V sup (ft o V/V 



c 



Proof. It is known that U(x) d = E x \£ ( j e =o < f> ° v (®k)\, where ac 
mi\k > 0,$fc £ C}, solves the equations PU(x) = U(x) — (ftoV(x), x^C and 
l/(x) = 0o V(x), x £ C [see Meyn and Tweedie (1993), Theorem 14.2.3]. By 
Proposition 4.1, U(x) < M~ 1 V(x) for all x £ C. Hence, 



clef 



(33) 



PU(x)<U(x)-4>oU(x), x$C. 



From (33) and Douc et al. (2004), Proposition 2.2, E^E^o* r^(fc)] < U(x) < 
M~ 1 V(x), for x C. The proof is concluded by noting that for x £ C, 



E, 



'TO — 1 

E 

. k=0 



<ri(0)+E x 



1{<MC} E r #) 

fc=i 



< r t(0) + M^ 1 sup PV(ar) < oo. 

V x&C 



□ 



Theorem 4.3. Suppose that {& n }n>o is a phi-irreducible and aperiodic 
Markov chain. Assume that there exist a function (ft£C,a measurable func- 
tion V : X — > [1, oo), a stopping time r > 1, a constant e £ (0, 1) and a petite 
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set C C X such that 

Pr-l 



(34) 

(35) 
(36) 



E, 



k=0 



< V(x), 



x<£C, 



E x [V($ T )]<(l-e)V(x), x(£C, 

sup{V + PV} < oo. 

c 



P is then positive Harris recurrent with invariant probability ir and: 

(1) for all x£X, \\m n — >00 r^(n)\\P n {x,-) — vr||xv =0, where (ft is defined 
in (32); 

(2) for all xeX, ]im n - HX \\P n (x, •) - tt\\^ v = 0; 

(3) the fundamental kernel Z is a bounded linear transformation from 
Lisy to L}L . 



Proof. (1-2) By Tuominen and Tweedie [(1994), Theorem 2.1], it is 
sufficient to prove that 



sup E a 



re — 1 

E 

. k=0 



< OO, 



sup E x 

xec 



E 4ov{* h ) 

. k=0 



< OO 



and, for all i£X, 
E 



TQ — 1 

E ^(k) 

k=0 



< oo, 



E 3 



E <t>°V($ k ) 

. k=0 



< oo. 



In Proposition 4.2 we show that the stated assumptions imply such bounds. 

(3) By Glynn and Meyn (1996), Theorem 2.3, it is sufficient to prove that 
there exist constants b, c < oo such that for all x € X, PW{x) < W(x) — cfto 
V(x) + btc(x), with W(x) < cV(x). This follows from Proposition 4.1, which 
shows that sup^E.E^o^ ^^)] <oo and E^E^o <f>°V(Qk)] < cV(x) 
for all x i C. □ 

Using an interpolation technique, we derive a rate of convergence associ- 
ated with some g-norm, < g < (ft o V . 

Corollary 4.4 (Theorem 4.3). For any pair of functions (a,j3) satis- 
fying a(u)(3(v) < u + v, for all (u, v) S HL + x M + and all x S X, 



lim a(r^(n))\\P n (x, •) - vr|| /3(<?ioV)vl 



0. 



A pair of functions (a, (3) satisfying this condition can be constructed by 
using Young's inequality [Krasnosel'skij and Rutitskij (1961)]. 
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4.2. Proof of Theorem 1.2. We preface the proof with a preparatory 
lemma. For any process {ek}k>i, define 



(37) 



M»(e,n) = f sup 

KKn 



E e * 

k=l 



Lemma 4.5. Assume Bl and B2. 

(i) For all k > 0, J and K integers with J < K , 

sup |$ fc+j -$ fc | 

0<k<k+j<K,0<j<J 

< 8M oc (e, K) + 2N(f3, A)k~PJ + N(P, A) + 2k, 

where N(/3,A) is given in B2. 

(ii) For all < a < /3 and all T > 0, i/iere exists M such that 



lim sup] 



sup 



>Mr ) =0. 



r ^°°xGX Vo<fc<fc+j<|Tr 1 + Q 

(iii) For all T > and e > 0, there exists 5 > suc/i i/iaf 



lim sup I 



sup 



<S> k +j - $ k \ >er)=0. 



Proof, (i) Let < j < J and < k < K - j. On the set rij=o{|$fc+j| > 



k+j-i 



(38) 



< 



< 



k+j 

E « 

l=k+l 
k+j 

E « 

«=fc+i 



fc+j-i 
+ E i A (^)i 

fc+J'-l 

+ 53 i^r^(AA) 



< 2M 00 (e,K) + JK- p N(P,A). 
Consider now the case when I < k for some < I < j — 1. Define 



Tj d = inf{0 < Z < j - 1, |$ fc+/ | < k} 



and 



ct-j = f sup{0 < I < j - 1, <k} + 1, 
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which are, respectively, the first hitting time and the last exit time before 
j of the ball of radius k. Write <S> k+j - <S> k = {<S> k+j - $ fc+(Jj ) + - 
& k+Tj ) + (& k+Tj — and consider the three terms separately. The first 
term is nonnull if a a < j; hence, 



I* 



k+j 



k+c 



< 



k+j 

E « 

l=k+Vj+l 



k+j-1 

+ E i A (^) 



l=k+<jj 



<2M 00 (e,K) + J K - p N(p,A) 

since, by the definition of dj, \<&k+l\ > K f° r an crj < I < j — 1. Similarly, for 
the third term, 



I* 



(39) 



E * 

l=k+l 



+ E i A (^ 



l=k 



<2M 00 (e,K) + J K - l3 N(p,A) 



since, by the definition of Tj, |<&/| > k for all < / < Tj. Finally, the second 
term is bounded by 



_i| + |$ fe+ _._i| + |$ fc+T .. 



<iV( / 3,A) + 2M 00 (e,^)+2K. 

Combining the inequalities above yields the desired result. 

(ii) From the previous inequality applied with k = Ir > and K = J 
[Tr 1+a \ , it holds that 

sup -$ fc | >4Mr 

0<k<k+j<[Tr 1 + a ] 
< 4 p M- p r- p supE :E [Af^(e, [Tr 1+Q J)] 

+ l{iV(/3, A) > Mr} + !{2iV(/3, A)T > ^Mr~ Q+/3 } + 1{2^ > M}. 

By Lemma A.l, the expectation tends to zero uniformly for x € X. The 
second term tends to zero when r — > oo. The remaining two terms are zero 
with I and M chosen so that l 1+(S > N(J3, A)T and M > 21 

(iii) The proof follows similarly upon setting K = [Tr 1+/3 J , J = K5r 1+/3 J 
and k = Ir. □ 



Proof of Theorem 1.2. Let a < (3. A sequence of probability mea- 
sures on D(R + ,X) is said to be D(R+, X)-tight if it is tight in D(R+,X) and if 
every weak limit of a subsequence is continuous. By [Billingsley (1999), Theo- 
rem 13.2, (13.7), page 140 and Corollary, page 142], the sequence of probabil- 
ity measures {Qr n] xJn>o is C(R+, X) -tight if (a) lim a _ +00 limsup n Q° n . !rn {77: |7?(0)| > 
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a} = 0, (b) limsup n ^Q? n . Xn {r7:sup 0<t < T \r](t)-ri(t-)\ > a} = and (c) for 
all k > and e > 0, there exist 5 € (0, 1) such that limsup n Q" n . a , ?i {r/ : w(rj, 5) > 

e} < k, where w(t},6) d = sup < s < t < T ,| t _ s |<,5 \v(t) - v(*)\- Properties (a)-(c) 
follow immediately from Lemma 4.5. Choose a < f3. Let {r n } and {x n } be 
sequences such that lim n r n = oo and lim n x n = x. Let e > 0. We have, for 
all n sufficiently large that \x n — x\ < e/2, 

p r„x n ( sup \rff n {t\x n )-x\ >e) <f rnX A sup \$ k - r n x n \ > (e/2)r n I 

\0<t<T J \<k<\Tr^ a \ / 

and we have (b), again by Lemma 4.5(h). □ 

4.3. Proof of Theorem 1.4. We preface the proof by establishing a uni- 
form integrability condition for the martingale increment sequence {e k }k>i 
and then for the Markov chain {& k }k>o- 

Lemma 4.6. Assume Bl. Then, for all T > 0, 

lim sup \x\-PE x [M^(e, LT|$ | 1+/3 J)l{M oo (e, m$ | 1+/3 J ) > b\$ Q \}} 
(40) b ~*°°\x\>l 

= 0. 

PROOF. Set T<j, d = [T\<S> \ 1+ P\ . For K > 0, set e k d = e k t{\e k \ < K} and 

£fc = f £ fcl{| e fc| > By Lemma A. 2, there exists a constant C (depending 
only on p) such that 

E a ,[AfS J (e,r* )l{M oo (e,r* ) > 6|* |}] 

< CE !B [JWS J (e,r* )l{M oo (€, T* ) > (6/2)|$ |}] +^[^(^,1,,,)]. 

Consider the first term on the right-hand side of the previous inequality. 
Using Lemma A. 3 with a > 1 V 2/p and Lemma A.l yields 

\x\-PE x [M^(e, Lr|<^ 1+ *J)l{M M (e,T* ) > (b/2)\* \}] 

< (6/2)~( a - 1 ) p |x|- ap E a; [M^(e,r ( i, )] < CA(e, ap)6 _( °- 1), '|xr o(1 -^ 1,/2 , 

where j4(e,ap) = f sup a , gX IE a ;[|ei| ap ]. Note that, by construction, A(e,ap) < 
K a P. Similarly, Lemma A.l implies that E x [M£,(elir,T$ )] < CM(eK",p) x 
T p/2| x |{p(i+/3)/2}v(i+/3) 5 where A(eK,p) d = sup^E^ei^]. Therefore, since 
p > 1 + 0, sup^i^! |z|- p E x [M£(eiT, < CT p / 2 A(eK,p). Combining the 
two last inequalities, we have 

sup |x|-PE a [JWS,(€,T* )l{M 00 (e,T fto )>6|*o|}] 

|x|>l 

< Cjiir^ft^^- 1 ) + A(eif,p)}, 
which goes to by setting K = f K(b) = log (6). □ 
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Proposition 4.7. Assume Bl and B2. Then, for all T > 0, 



(41) 



sup(l + \x\)' p E x 



sup 

0<fc<|T|* | 1+/3 J 



< oo, 



(42) 



lim sup |x| p E a 

K -*°°\x\>i 



sup \§ k \ p 

'0<fc<|T|$ | 1+/3 J 



sup \<&k\ > K\$o\ 

Lo<fc<|T|* | 1+/3 J 



0. 



Proof. Set T$ = [T|$ | 1+/3 J- For all r > 1, applying Lemma 4.5(i) 
with K = J= LT|$ | 1+/3 J and k = |$ | yields 



(43) 



sup |* fc | r <C7{l + |*o| P + A^(c,r* )} 

0<fc<Tb Q 



for some constant C depending upon r, /?, N(j3, A) and T. The first assertion 
is then a consequence of Lemma A.l. Inequality (43) applied with r = 1 
implies that there exist constants a, b > such that for all \x\ > 1 and all 
large enough K, 



( sup \§ k \>K\§ \\c{M oo (e,T* )>(aK-b)\$ \} 

lo<fe<Td,„ J 



-a.s. 



.0<fc<T $0 

Hence, for large enough K and an appropriately chosen constant C, 



sup \x\ P K X 

\x\>l 



sup |$fc| p ]H sup |$ fc |>if|$ | 
< C sup P x [M oo (e,T <I)0 ) > (aK - 6)|$ |] 



\x\>l 

+ C sup |z|-PE a [Af* (e, T* )l{M oo (e, T$ ) > (oif - b) | $ 1 }] • 

|x|>l 

The proof of (42) follows from Lemma 4.6. □ 

Proposition 4.8. Assume Bl and B2 and that there exist T < oo and 
p £ (0, 1) such that 

(44) limsupP s (o->T) =0, with a d = inf{A; > 0, < p|$ |}, 

| scr | — >oo 

where r = ctA [T| ( l ) o| 1+/3 l • then follows that (a) i/iere exists M such that 

su P|x|>Af kl _P IEa;[| < ^r| P ] < 1 

(b)~E x [j:iZ l \^ k \ p ]<C\x\P+ 1 +' 3 . 
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PROOF. Set T$ = [T|$ | 1+/3 1 • For any K > 0, 

(45) = |x|-fE x [l{r = (7}|* T H + \x\-m x [l{a > r* }|*r./] 
<ff + \x\- p M x [\ $ T#Q | p l{| $ T$o \>K\$ \}} + K P F X [a > T $0 ] . 

By Proposition 4.7, one may choose K sufficiently large so that 

(46) sup |x|- p E x [|$ T | P 1{|$ T | > #|$ |}] < 1 - 

|x|>l 

Since limsupi^i^^Pajfo" > T$ ] = 0, the proof of (a) follows. Since r < T$ , 
(b) follows from (41) and the bound E x [£fc=o |$fc| p ] < CT|s| 1+ ^ x 

Ex[sU Pl < fc < T4jo □ 

The following elementary proposition relates the stability of the fluid limit 
model to the condition (44) on the stopping time a. We introduce the polyg- 
onal process that agrees with $fc/r at the points t = fc r ~( 1 + a ) and is defined 
by linear interpolation 

fj?(t; x) = r" 1 ]T{(£: + 1 - tr 1+a )<£ k + (tr 1+a - k)<Z> k+1 } 

(47) k>o 

xt{k<tr 1+a <(fe + l)}. 

Denote by Q^. x the image probability on C(R + ,X) of F rx by fj^(t;x). The 
introduction of this process allows for an easier characterization of the open 
and closed sets of C([0, T],X) equipped with the uniform topology, than the 
open and closed sets of D([0,T],X) equipped with the Skorokhod topology. 
For any sequences {r n } n C K + such that r n — > +oo and {x n } C X such that 
x n — ► x, the family of probability measures {Qr n -. x „} is tight and converges 
weakly to Q x , the weak limit of the sequence {Q^ n . Xn } n ^. This can be 
proved following the same lines as in the proof of Theorem 1.2 [see, e.g., 
Billingsley (1999), Theorem 7.3]. Details are omitted. 

PROPOSITION 4.9. Assume Bl, B2 and that the (3-fluid limit model 
{Qf,xGX} is stable. Then, (44) is satisfied. 

Proof. Let {y n } C X be any sequence of initial states with \y n \ — > oo 
as n — > oo. Set r n = \y n \ and x n = f y n /\Vn\ - One may extract a subsequence 
{ x nj} Q {%n} such that Hindoo x nj = x for some x, \x\ = 1. By Theorem 
1.2, there exist subsequences {r mj } C {r nj } and {x mj } C {x n } and a /3-fluid 
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limit such that Q(? . x =^Q^. By construction, 



(<J > r) < F rm . Xm . ( Q mf T |C. (*;^)l > P) 



By the Portmanteau theorem, since the set {77 G C(M + ,X),inf[o i T] \v\ — p} 1S 
closed, we have 

limsupQ^,^. ( o mf T \rj(t)\ > p) < Qg ( Q irf, |*t)| > p) = 0. 

Because {y n } is an arbitrary sequence, this relation implies (44). □ 

Proof of Theorem 1.4. This follows immediately from Theorem 4.3, 
using Propositions 4.8 and 4.9. □ 

4.4. Proof of Proposition 1.5. In this proof, we see the /3-fluid limit Q@ 
as the weak limit of Q£ for some sequences {r n } C R+ and {x n } C X 
satisfying lmin^oorn = oo and lin^^oo^ = x. Fix s,t such that s < t. We 
prove that 



(48) 



A(a,i)n^eC([a,i],X): 



sup 

s<n<t 



77(11) -r/(s) - / hor](y)dy 



>0 =0 



Let U be an open set such that U C 0, where U denotes the closure of the 
set U. For any 5 > 0, M > and m> 0, s < u < w < t, define 

A 5,m,M(^ «>) = \v G C([s,t],X),7?([n,w]) C UnC m , M , 

(49) 



sup 

M<t;<iu 



i](v) — rj(u) — / hor](x)dx 



>8 



where C mj M = {i£X,ra< |x| < M}. Since e), m, M, U, ti and w are arbi- 
trary, (48) holds whenever Q^[A^ m M (u,w)) = 0. By the Portmanteau theo- 
rem, since the set A)- mM (u,w) is open in the uniform topology, 

Of KmM u > w )] < l™infQf„ ;:c jA^ m)M (u,«;)] 
and the property will follow if we can prove that the right-hand side of the 
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previous inequality is null. To that end, we write 

Vr n iv; x n ) - r$ n (u;x n )- ho fj^ n (y; x n ) dy 

Ju 

= fj? n (v; x n ) - ff rn ( \vr\^\ ; x n ) 

+ € ( lur^\ r~( 1+ « ; x n ) - ff Tn 

+ r~ l E {®k+l-$k}- J hof}? n (t;x n )dt 

< 2xi + X2 + Xs + 2r" 1 M 00 (e, K +/3 J), 
where we have defined 



def 

Xi = sup 

u<v<w I 



( I C (v-x n )- fji ( [vr 1 ^ j ; x n ) | 

hofjP(t;x n )dt 



+ 



1+0 I ^ J r n 



X?> 



j=|w n +p j 



Denote by uj m ,M,\3 the modulus of continuity of h on U n C m> M- Since /j 
is continuous on U, ]imx-^o^>m,M,u(^) = 0. On the event {fj^ n (t;x n ) 6 U n 

XI <^n 1 (l+ SUp |/l(x)|) SUp l^+i 1, 

^ |ac|>m / l<j<[tri +l3 i 

X2<(t- 8 + l)m~ p sup |r^|a;|^A(r n a;)- Aoo(a;)| 

{a;GU,|x|>m} 

and, for any A > 0, 

X3 < (t-s + l)( w m ,M,u(A)+ sup sup \$ j+1 -®j\>\r n \). 

\x\>m ^l<j<\trl+ p \ 



By Lemma 4.5, for any 5 > 0, lim fl _ 4 . 00 P r . na!n (sup 1 ^.< Ltr i+/jj - $j\ > 

5r n ) = 0. On the other hand, ]im n -> 00 sup^gu^^^j | \x\^A(x) - A 00 (x)| = 
0. Therefore, for any J > 0, one may choose A small enough so that 

lim F rnXn (ftf n (t; x n ) £ Ufl C m , M , (2xi + X2 + Xs) > <?) = 0. 
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The proof follows from Lemma A.l. 

4.5. Proof of Theorem 1.6. We preface the proof by a lemma showing 
that the fluid limits are uniformly bounded. 

Lemma 4.10. Assume Bl and B2. 



(i) For any T > and p > 0, there exists 5 > such that, for any (3 -fluid 
(50) Qdr ] €C(R + ,X), sup \n( U ) - n(t)\ < p) = 1. 



limit QP, 



0<t<u<t+S<T 

(ii) For any T > 0, there exists K > such that, for any (3 -fluid limit Qj, 
(51) L £ C(R+, X), sup \ V {t) - r/(0)| >k)=0. 

\ 0<t<T / 

Proof, (i) Let {r n } C M+ and {x n } C X be two sequences such that 
lim-n^ooTn = +oo, lim n ^ oc x n = x and Q£ . x =^ Qg. By the Portmanteau 
theorem, since the set {77 G C(M + , X), sup 0<t<u<t+( 5< T |^(u) — ??(t)| < p} is 
closed, it follows that 



\( V eC(R+,X), sup \v(u)-ri(t)\<p) 

\ 0<t<u<t+S<T / 



>limsupQ^ ;a; Jr ? eC(]R + ,X), sup \r,{u) - n(t)\ < p 

n \ 0<t<u<t+8<T 

By definition of the process fjf n (-;x n ), 

®?( V eC(R + ,X), sup \r,(u)- V (t)\>p) 

\ 0<t<u<t+S<T / 

< K n x n ( sup I <S> k+j -& k \ > pr n 

^O^kKk+j^Trl^ ,0<j<Sri +l3 

and the proof follows from Lemma 4.5(iii). 

(ii) The proof follows from (i) by considering the decomposition 

[T/8] 

sup \rj(t) -77(0)| < SU P l»7( u ) -»7(? 5 )l- 

0<i<T Q qS<u<(q+l)8 □ 

Proof of Theorem 1.6. Under the stated assumptions, p([0,T x ];) is 
a compact subset of 0. Since O is open, there exists p > such that 

{y eX,d(y,p([0,T x ];x)) <2p} CO, 



THE ODE METHOD FOR MARKOV CHAIN STABILITY 33 

where, for x E X and AcX, d(x,A) is the distance from x to the set A. By 
Lemma 4.10(i), there exists 5 > such that 

'r/GC(M+,X), sup |i7(u)-f7(t)|<p) =1. 

V. (Kt^lKt+cKT.,, / 

Since Q^(?? E C(M+,X), 77(0) = x = p(0;x)) = 1, we have 

Qf( ?? EC(R + ,X),7 ? ([0,5])cO) = l. 

By Proposition 1.5, this yields = <^(.; X ) on C([0,<5],X). By repeated appli- 
cation of Lemma 4.10(i), it is readily proved by induction that Q@ = Sp(.- X ) 
on C([(q - 1)6, q5] n [0, TJ , X) for any integer q > 1. □ 

4.6. Proof of Theorem 1.9. Let x be such that |x| = 1. By Lemma 4.10, 
there exists K depending on To such that Q^(r/ : sup[ 0) r ] < if) = 1 for 
any /3-fluid limit Q^. Set T = Tq + Tr-, where To and Tk are defined by (11) 
and (12), respectively. 

By definition, for any set H, H C Oh; therefore, there exists an increasing 
sequence {H n } of compact subsets of such that H n C H n+ i and = \J n Q\-\ n 
(note that Q\-\ n C £l\-\ n+1 ). This implies that 



r,:mi\r,(-)\>p K , V ([O,T o ])nO^0 

= lim |n Qf inf |r?(-)| > pkMM]) 

lim |„ stands for a limit that converges monotonically from below. We prove 
that for any n, the term in the right-hand side is zero. To that end we start 
by proving that for any compact set H C and any real numbers < q < Tq , 

77: inf |??(-)| > p K ,ri(q)en H 

(52) =qd V :M\ V (-)\>pKM<l+-)=P(--M<l)) 

on [0,T,( ff )], 77(g) E f^H ) • 

We will then establish that 

(v : inf KOI > PAT, r/(g + = M"! »/(«)) 

(53) 

on [0,T v{q) ],n(q)en H )=0. 
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Since Q^(C(M + ,X)) = 1, (52) and (53) imply that 
Qf (n : mf H-)\>PK, r?([0, T }) nQ Hn ^0 

<Y,^x(v-^M-)\>PK,ri{q + -)=fi(-;r l {q)) 
q eQ v 10,1 



on [0,T ri[q) ],r](q) en Wn J =0, 

where D H n is a compact set of and Q C [0,7o] is a denumerable dense 
set. This concludes the proof. 

We now turn to the proof of (52) and (53). Since fin is a compact set of 
0, there exists e > (depending on H) such that {y £ X,d(y, f2n) < 2e} C 0. 
By Lemma 4.10, one may choose 5 > small enough (depending on T and 
e) so that 



'(i/eC(R + ,X): sup \rj(u) -rj{t)\ < e) = 1. 

V 0<t<u<t+5<T / 

Therefore, for any compact set H C and q £ Q, 



t[r}--^fM-)\>p K ,r ) (q)en H 



= Qg (77: inf |r?(-)| >p K ,v(<l) S^h, sup |r?(n) - »/(t)| < e 

V [0,T] 0<t<u<t+5<T 

= Qf(r? : inf \ V (-) \ > p K , 77(g) € fi H ,^([g, (? + S) A T]) C 0) . 

By Proposition 1.5, on the set A(q,q + 5), T}(q + -) = p{-]f]{q)) 011 [0,MT^ g )], 
O^-a.s. Hence, 



^ : i n | >pk,v(<i) e H , 77(9 + ■) =/i(- 577(g)), 

on [0,5 A T^)] 

By repeated application of Proposition 1.5, for any integer I > 0, 
Q^ jnf IOI>^,^)€0 H 

77: inf |??(-)| > Pk,v(q) £fl H ,r)(q + ■) = /j,(-,r){q)), 
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on [0,16 A T^] 

which concludes the proof of (52). 

t£[ri- mf |ij(-)l>«r,l(« + -)=l'(-;i(«)) on [0,T, ( ,)],,,(,) e !J„ 



sS ra w,]l> '"*KH WStt ) =1 

since T = To + T&-, which concludes the proof of (53). 
5. Proofs for Section 2. 

5.1. Proofs of Section 2.1. 

Proof of Proposition 2.4. Define 

(54) A(x) d ^-[ yq(y)X Lc \dy). 

Introduce, for any 5 > 0, the 5-zone C x (5) around C x , 

(55) 0,(5) = {y + sn(y),y eC x ,—6<s< 5}. 

By Jarner and Hansen [(2000), Theorem 4.1], we may bound the measure 
of the <5-zone's intersection with the ball B(0, K), for any K > and all |sc| 
large enough, 

A-(C(fl n B(o. A-)) ^Mll pW")! , 

where the x-dependent term tends to 1 as |x| tends to infinity. From this, it 
follows, using the fact that / \y\q(y) X Leh (dy) < oo, that for any K > and 
e > 0, there exists 5 > such that 

(56) limsup/ M<7(y)A Leb (A/) < e, 

|a-|-*oo JEx(8,K) 

where E X (S,K) d = C x (6) n B(0, if). For arbitrary, but fixed, e > 0, choose 
K > such that JB c (oif) |yk(y)^ Leb (^?/) ^ e - Then choose 5 > such that 

(56) holds. By construction, for y € R x , 7r(x + y)/n(x) < 1 and (56) implies 
that 

(57) limsup / |y| 7r(X , + ^ ) g(y)A Lcb (^) < e, 

(58) hmsup / \y\ 7rix ^ y) q(y)X Lch (dy) < e. 
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From (15), for y G R x such that y has radial distance at least 5 to C x , the 
acceptance probability satisfies tt(x + y)/ir(x) <e/K for all \x\ sufficiently 
large [see Jarner and Hansen (2000), page 351] and (56) shows that 

(59) limsup / \y\ 7r{X f \ y) q(y)X Leh (dy) < e. 

By combining (14), (54), (57), (58) and (59), limsup^i^ \A(x)-A(x)\ < 3e 
and since e is arbitrary, limi^i^oo |A(x) — A(x)| =0. □ 

Proof of Proposition 2.6. Set z = (z±, . . . ,z d ) d = XT 1 / 2 ?/ and v = 
n (E^u). Then, 

yq(y)X Lch (dy) = E 1 / 2 f zq (z)X Lch (dz) 
{y,y'u>0} J{z,v'z>0} 



= £ 1/2 t> / z 1 t {zi > 0} q {z)dz. 

The proof follows. □ 

5.2. Proof of Lemma 2.9. Let 5 and M be constants to be specified later. 
Write A(x) - A m (i) d = J2t=i Ai(5,M,x), where 

Ai(5, M,x) d ^ f / T±±yl t {y ) yq (y)X Lch (dy), 

A 2 (5,M,x) d M f i (^j^-l)(lRM-t Roo ,M) 

Hy,\y\<M,\y'r2 x\>s\x\}\ n{x) J 

xyq(y)X Lch (dy), 

A 3 (6,M,x) d ^ f {(^j^-AtnM + ^M) 

J{y,\y\<M,\y'r- 1 x\<5\x\} I V tt{x) J J 

xyq(y)\ Lch (dy), 

A^M,x) M f { f^E + Vl - lW(y) + t Roo Jy) )yq(y)X Lch (dy). 

J{y,\y\>M} I V nx) J J 

For x = (x\,X2) such that — \x 2 \ > 2M and \y\ < M, and \xi + yi| > 
\x\ \ — M > \x 2 \ + M > |x2 + 2/2 1 5 it is easily shown that 

(l-a)exp(-0.5y , r 2 - 1 y-x'r2 1 y) 
7r(x + y) 



(60) < 



n(x) 

< (1 - a)" 1 exp(-0.5y / r^ 1 y - x'T^y). 
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If V € Roo )X n{z: \x'T^ l z\ > 5\x\}, then, by (60), ir(x + y) /vr(x) < {I - a)~ l e~ 5 \ x \ 
which implies that \A\(5, M,x)\ < (1 — a)~ 1 e~ 6 ^ J \y\q(y) X Leb (dy) . Further- 
more, for any K such that (1 — a)~ 1 e~ SK < 1 and x such that ||xi| — \x2\ \ > 
2M and \x\ > K, R^ x n {y : \y\ < M, {x'T^yl > 6\x\} C R x . This property 
yields to the bound 

vr(x + y) 



7r(x) 



1 



Wy) - ifloc,,(y)|i{y, Kr^ y | > <M | y | < m} 



(61) 

< *&vw (y)Hv, \y\ < m, \x'r^ y \ > s\x\}. 

Again using (60) for yeR x D {\y\ < M }, (1 - a ) e ~0-5a 2 M 2 e -x'r^ y < ^ + 
y)/ir(x) < 1. On the other hand, for j/ ^ .Rcx,^ satisfying jxT^" 1 y| > 5|x|, we 
have x'r^" 1 ?/ < — 5\x\, showing that 

y £ R x \ Roo, x n {z, \z\ < M, \x'r^z\ > 5\x\} 

(1 - a)e~°- 5a2M2 e 5K < ir(x + y)/ir(x) < 1. 

For fixed M, we choose K such that (1 — a)e~°' 5a " 2 m2 e &K > 1, which implies 
that the right-hand side in (61) is zero and thus A2(S,M,x) = 0. Finally, 
consider Ai(S,M, x), i = 3,4. Noting that 

'ir(x + y) „\ , . , . 

-l )t Rx (y) + ^RooJy) 



< 2, 

TT{X ) J 

the proof follows from the bounds 

(62) \A 3 (5,M,x)\<2M J t{y, \y'Y^x\ < 5\x\}\y\q(y) X Lch (dy) , 

(63) \MS,M,x)\<2 [ \y\q(y)X Lch (dy). 

J\y\>M 

These terms are arbitrarily small for convenient constants M and 5. 
5.3. Proof of Proposition 2 AO. 

5.3.1. Proof of condition (i) of Theorem 1.9. The only difficulty here 
stems from the irregularity of the ODE for initial conditions on the diago- 
nals. Consider the /3-fluid limit with initial condition u± = f (l/\/2, 1 / \/2) 

(the other cases can be dealt with similarly). Set v+ = f (l/y/2, — l/y/2) and 
define V(x) = \(v i! ,x)\. Since the increment distribution is assumed to be 
bounded, there exists a positive constant C q such that |$i — 3>o| < C q , P x - 
a.s. for all x £ X. By Lemma 2.9, we may choose constants 7 £ (0, 1), m > 0, 
Mo > C q and such that 

(64) Rn E c c {x G X, \ (v in A(x))\ > m, (v*,x)(v±, A(x)) > 0}, 
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Fig. 9. The complement of the zone R and the strips E and F. 
where (see Figure 9) 

(65) E d = {x, V(x) < M } and R d = {x G X, \x\ > R, \(v+, n(x)}\ < 7}. 

For 5 > 0, define the stopping time k(5) as the infimum of the foliowing 
three stopping times 

dcf . 



«x(<5) =inf{fc>0,|K,$ fc )|>2<5|$o|} ) 
k 2 = mf{k > 0, |* fc - $ | > (l/2)|*o|}, 

def . 



(66) 
(67) 

(68) k 3 u = inf{A;>0,|$ fc | <^}. 

We will establish the following drift condition: there exist constants b > 
and C such that for all 5 G (0,7/4), 

(69) E[V($ k+1 )\F k ] >V($ k )+m-bl E (*k) on the set {fc < k(<5)}, 



(70) E, 



Tt(<5)-1 

fc=0 



<c, 



with the convention that J2 a = when a> b. We postpone the proof of (69) 
and (70) and show how these drift conditions allow us to obtain condition 
(i). On the event {k < k(S)}, |$ fc | > R, (1/2)|$ | < |*fc| < (3/2)|$ | and 
|(u*,n($fc))| <45<7. Therefore, for all x G X, Pa.-a.s-, 

(71) {A; < k(S)} C G R}. 

Condition (69) yields, for any constant N > 0, 

-«(5)AAT-1 

wE x [k{5) AN]<E x [V($ KmN )HK(5)>l}}+bE x ]T M^fc) 

. fc=0 
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The definitions of k(5) and C q imply that K x [V(^ k ^) A n)H k (^) - 1}] - 
25\x\ + C q for all N, which, with (70), yields the bound 

(72) mE x [K(8)]<26\x\+bC + C q . 

Let {x n } be a sequence of initial states such that linv^oo x n = u* and {r n } 
be a sequence of scaling constants, linin^oo^ = +oo. By Lemma 4.10, there 
exists To such that Q^{sup te r To ] \rj(t) — r/(0)| < 1/4} = 1. Furthermore, we 
have 1/2 < \x n \ < 3/2 for all n large enough. Then, by the Portmanteau 
theorem, 

Ql{^([o,T o ])no = 0} 



= HmQl <T), sup |77(t) - 77(0)| < 1/4, sup \(v„r](t)}\ < 5 
<no + I te[o,T ] te[o,T ] 

< lhnliminfP^ I sup |$ fc - $ | < (1/2)|$ |, 

SIQ+ n->oo lo<fc<2T |* |/3 



sup \(v*,<S> k )\<25\$ \) 

:2Tnl*nl/3 > 



0<fc<2T |$o|/3 

< lim liminf P rnX „(/e(5) > 2T |$ |/3) = 0, 

<5J,0+ n -*°° 

where the last equality stems from (72). This proves Theorem 1.9(i). 

We now prove (69). Since E[<I>j! c+1 |jr fc ] = <E> fe + A($fc), Jensen's inequal- 
ity implies that E x [V($ k+1 )\F k ] > \(v*,$ k + A(* fc ))|. Furthermore, by (64) 
and (71), {A; < K(8),$ k G E c } C {& k G Rn E c }, which implies that | + 
A(® k ))\-\(v*,<f> k )\ = \(v*,A(® k ))\>m since, onRnF, (v±,x) and (u*,A(z)) 
have the same sign and (v*, A (a;)) is lower bounded. On the set {k < k(5), & k G 
E}, we write V($ k +i) > V(* k ) - C q so that E[V($ k+1 )\F k ] > V{<& k )+m- 
(C q + m). This concludes the proof of (69). 

Finally, we prove (70). For A G X, we denote by a A = inf{k > 0, G A} 
the first hitting time on A For notational simplicity, we write k instead of 
k(5). Define recursively = f u ehr and, for all k>2, = f a^ k ~^ + r o 
(9°" (fc x> + <7 W o ^ ro6 ' <T +°" <fc x) ; where r = f k A k*, fc* being an integer whose 
value will be specified later. With this notation, 

r«-i i 

<h^ X (<T {q) < 

Furthermore, for all q > 2, the strong Markov property yields the bound 



(73) E 



£1e(**) 
Lfe=0 



F x (aW <k)< F x (a^-^ < k) sup F y (r + (T (1) o6 t <k) 

j/eEnR 
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Therefore, by (73), (70) holds, provided that sup x . eEnR F x (t + cr (1) o T < 
k) < 1. For all x G E n R, it is easily seen that 

F x {t + ct (1) o6 t <k) 

(74) 

= F x (t < k) - E x (1{t < k}1{$ t G E c n R}P$ T [k < a {1) }) 

(75) <1- inf F x (K<a^){F x (T = n)+F x (T = k*,$ k ^£E c nR)}, 

xeE c nR 

showing that the conditions 

(76) inf P I ({r<Mu{r = ^f fct eE c nR})>0, 
zeEnR 

(77) inf F x (K<a {1) ) >0 

imply (70). We first prove (76). Choose 7 G (7,1) such that the four half- 
planes {z, (z, T" 1 ^-) < 0} (i = 1,2) have a nonempty intersection, where 

and n^- are the unit vectors defining the edges of the cone C^, = {z G 
X, \(v*,n(z)) <7}. Define 

(78) W d ^ f {z, < |z| < C q , (z, rr 1 ^.) < 0, i = 1, 2}. 

Since any vector y in the cone can be written as a linear combination of 
the vectors u~~ and with positive weights, for any j/£C^ and z G W, 
(zjT^ 1 ?/) < 0, i = 1,2, which implies that 

(z,W(y)> 

= -a(z,rj" 1 7/) exp(-0.5?/ / rj" 1 y) - (1 - a)(z,rj 1 y) exp(-0.5y'r2 X y) 
>0. 

By choosing i? large enough [see (65)], we can assume, without loss of gen- 
erality, that for all x G R and z G W, x + tz G Oy for all £ G (0, 1). Thus, 
tt(x + z) = tt(x) + J 1 (V'7r(x + tz), z) dt > and we have 7r(rr + z) > 7r(x), 
showing that W C A x . Finally, we write W as the union of two disjoint sets 

W~, W+, where W+ d = {z G W, (v*,z) > 0}. Since, for x G R, W C Ax, for 
any < c < C q , we have 

inf P s (|(t;*,$i>|>K«*,*o>|+c) 

seR,(«*,a;)>0 

> / l{y,|(^,y)|> C My)A Lcb (dy)>0. 

J\N+ 

An analogous lower bound holds for all x G R such that (i>*,x) < 0. These 
inequalities, combined with repeated applications of the Markov property, 
yield (76), by choosing k* such that k+c > Mq. 
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. def 
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We now prove (77). Let M\ > Mq and set F = {x, V(x) < Mi}. By Lemma A.l, 
we may choose J > 1 and then Mi > Mq large enough so that, for all x G X, 



x\ sup j 

\j>J 



j 

E e < 



x- SUp 




> Mi 




Z=l 





>mj < 1/2, 
>Mi -M ] < 1/2. 



(79) 



It is easily seen that, using the strong Markov property, 

inf F x (n<a E nR)> inf F x (a F c nR < cjehr) inf < cr EnR ). 

o;GE c nR xGE c nR xGF c nR 

The first term of the right-hand side of the previous relation can be shown 
to be positive, using arguments which are similar to those used in the proof 
of (76). We write (u*, $ fc ) = $„) + Ef=iK, + Ef=i(«*, £/)■ Let 

x G F c n R. P-c-a-S., since - < C g < M , on the event {1 < A; < 

^EnR<^}, | («*,**} | > M , (v*,$ )K,%}>0 and («*,*„}(«*, A^-)) >0 
for all < j < k, which implies that 



|K,$ fe )|>|K,$o)|+ElKA(^_i))| 



E( u *' e ^ 



> Mi + km 



E^*> e *) • 
i=i 

Thus, for all x G F c n R, using the definition (79) of J and Mi, we have 



,{J < o- EnR < k} < supP x < supj" 

zGX I j>J 



j 

E^*> e ') 


> m 


i=i 





cjo'EnR < k A J} < supP x .< sup 
zex li<j 



E« 



> (Mi - M ) < 1/2, 



which proves inf xg F c nR Pr( K < ^edr) > and therefore (77). 

5.3.2. Proof of B4 and the conditions (ii)-(iii) of Theorem 1.9. As- 
sume that x G C = f {x,0 < | 1 < ^l} (the three other cases are similar). 
By Lemma 2.9, h(x) = —CqniT^x) for all x G C, which is locally Lips- 
chitz. Hence, there exists a unique maximal solution n(-;x) on [0,T X ] sat- 
isfying fi(0;x) = x and n(t;x) G C for all i < T x , showing B4. Since, for t G 
[0,T X ), d/dt\fi(t;x)\ 2 =2\fi(t;x)\(n(n(t;x)),ho n(t;x)) < -2c q \a\~ l \^{t;x)l 
the norm of the ODE solution is bounded by |/i(t; x)\ < (\x\ — c q \a\~ 1 t) + for 
all < t < \x\\a\c7 l , which implies condition (ii), provided that T x > Tk for 
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all x G C n B(0, K). This result follows from the fact that the boundaries 

of C are repulsive: consider the relative neig hborhood in C, V = Vi U V 2 , 

of the boundaries where Vi = f {x:x\ > 0, (v*,x) > 0, (x,^ 1 ^*) < 0} and 

V2 *== {x : x\ > 0, (u*,x) > 0, (x, r^u*) < 0}. Assume that there exists s G 
[0,T X ] such that fJ,(s;x) G Vi (the other case can be handled similarly). Since 
1 1— ► fj,(t; x) is continuous and Vi is a relative open subset of C, there exists 5 
such that for all < t < 6, /i(s + t;x) G Vi. This implies that for all < t < 5, 

fj,(s + t; x)) - {v*, fi(s; x)) 

= —c q / |r^~ n(s + u; /x(s + n; x)) du > 0, 

Jo 

showing that, in Vi, the distance to the boundary always increases. The 
properties above also imply condition (hi) of Theorem 1.9. 



APPENDIX: TECHNICAL LEMMAS 

Lemma A.l. Let {£k}k>i be an L p -martingale difference sequence adapted 
to the filtration {J~k}k>o- For any p > 1, there exists a constant C (depending 
only on p) such that 



(80) 
(81) 



E 



sup 

KKn 



k=l 



sup I 1 

n<l 



fc= 



< C supE[\e k \ p ]n 

k>l 



Plr.lVp/2 



k=l 



> M 



< CsupE[|e fc | p ]M-Pn- p+1Vp / 2 . 

k>l 



Proof. For p > 1, applying in sequence the Doob maximal inequality, 
by the Burkholder inequality for L p -martingales, there exists a constant C p 
such that 

n p/2h 



E 



sup 

KKn 



k=l 



< CpE 



k=l 



Equation (80) follows from the Minkovski inequality for p > 2, 



(82) 



E 





I 


P- 


sup 






KKn 


k=l 





< C p supE[|e fc | p ]n 

k>l 



P] n P/ 2 



and the subadditivity inequality for 1 < p < 2, 



(83) 



E 



sup 

KKn 



k=l 



< C p supE[|e fc | p ]n. 

k>\ 



Equation (81) follows from Birnbaum and Marshall (1961), Theorem 1. □ 
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4.3 



Lemma A. 2. Let X,Y be two nonnegative random variables. Then, for 
any p>l, there exists a constant C p ( depending only on p) such that, for 
any M > 0, 

E[(X + Yft{X + Y > M}] < C p (E[X p t{X > M/2}} + E[Y P ]). 

Proof. Note that t{X + Y >M}< 1{X > M/2} + l{X < M/2}1{Y > 
M/2}. Therefore, 

E(X p t{X + Y> M}) < E(X P 1{X > M/2}) + (M/2) P F(Y > M/2) 

< E(X p t{X > M/2}) + E(Y P ). 

The proof then follows from the fact that {X + Y) p < 2 p ~ l (X p + Y P ). □ 

Lemma A. 3. Let X be a nonnegative random variable. For any p > 0, 
a > 1 and M , we have 

E[X p t{X > M}} < M- ( - a -V p E[X ap }. 

Proof. 

E[X p t{X > M}] < (E[X ap ])^ a {P[X > M])^- 1 )/ 

< (E[X ap ]) 1/a (M~ ap E[X ap ]) ( - a ~ 1)/a . □ 
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